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I In this paper we analyze a specific class of rateless codes, called LT codes with unequal recovery 

. time. These codes provide the option of prioritizing different segments of the transmitted data over other 

^ I ' The result is that segments are decoded in stages during the rateless transmission, where higher prioritized 

. ' segments are decoded at lower overhead. Our analysis focuses on quantifying the expected amount of 

^ received symbols, which are redundant already upon arrival, i.e. all input symbols contained in the received 

symbols have already been decoded. This analysis gives novel insights into the probabilistic mechanisms 

' of LT codes with unequal recovery time, which has not yet been available in the literature. We show that 

00 ' while these rateless codes successfully provide the unequal recovery time, they do so at a significant price 

, in tcrms of redundancy in the lower prioritized segments. We propose and analyze a modification where a 

. single intermediate feedback is transmitted, when the first segment is decoded in a code with two segments. 

O ■ 

' Our analysis shows that this modification provides a dramatic improvement on the decoding performance 

• ' of the lower prioritized segment. 

> '. 

X: 

; I. Introduction 

Rateless codes are capacity achieving erasure correcting codes. Common for all rateless codes 
is the ability to generate a potentially infinite amount of encoded symbols from k input symbols. 
Decoding is possible when (1 + e)k encoded symbols have been received, where e is close to zero. 
Rateless codes are attractive due to their flexibility. Regardless of the channel conditions, a rateless 
code will approach the channel capacity, without the need for feedback. Successful examples are 
LT codes [HI and Raptor codes flU. 

Standard rateless codes treat all data as equally important. In some applications, e.g. video 
streaming E, flU, this is not desirable due to dependencies between data segments. Several works 
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address the problem of designing rateless codes which provide unequal error protection (UEP), 
where different data segments have different error probabilities when a certain amount of symbols 
have been collected. Equivalently they can also provide unequal recovery time (URT), which refers 
to the different amounts of symbols it requires for the different data segments to achieve the 
same error probability. Variants based on LT codes are found in [|5]|-[l9l|, while ifTOl is an example 
using Raptor codes. Common for all approaches is the idea of biasing the random sampling in 
the rateless code towards the more important data. The different works are distinguished by how 
this is achieved. A very simple yet elegant solution, which stays true to the original LT encoding 
structure is found in [|9l. This solution replaces the originally uniform sampling of input symbols 
with a nonuniform one, where more important symbols are sampled with higher probability than 
less important symbols. The authors provide asymptotic analysis of the proposed codes for belief 
propagation (BP) decoding and finite-length analysis for maximum- likelihood (ML) decoding. In 
both cases, the UEP and URT properties are shown to be provided. 

In this work, finite-length analysis is presented for BP decoding of the UEP/URT LT code from 
|l9l. We focus on the case of full recovery of the individual data segments, i.e. fixed error rate of 
zero. Thus, these codes are referred to as URT-LT codes. The purpose of this analysis is to quantify 
the amount of redundancy realizations of these codes introduce, when recovery of all data segments 
is desired. The analytical approach is based on a decoder state recursion, similar in structure to the 
one presented in [|TT|. though fundamentally different in its elements. To the best of our knowledge, 
this is the first finite-length analysis for BP decoding of this type of URT-LT codes. 

Through evaluation of our analytical results, we show that the improved decoding performance 
of the more important data is only achieved by significantly degrading the decoding performance 
of less important data. Motivated by this result, we propose and analyze a modification of these 
codes, where successful decoding of a data segment is reported to the transmitter through a feedback 
channel. Adding such an intermediate acknowledgment of data is shown to have great potential in 
URT-LT codes. The proposal to acknowledge only a data segment is deceptively simple, however, 
as it can be seen from this text, the analysis is very involved. Most importantly, adding such an 
intermediate acknowledgment is shown to have a great potential in URT-LT codes. The assumption 
of having additional feedback available during a transmission may be considered a strong assumption 
in many communication systems. Yet it should be noted that rateless codes require a single feedback 
message at the end of the successful decoding. Our analysis shows that it is very beneficial to add 
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one more such feedback message. On the other hand, often feedback is inherently available from 
the layers that run below the rateless code, such as e.g. the link-layer in cellular systems. 

The remainder of this paper is structured as follows. Section |ll] gives an introduction to LT 
codes, explaining the encoding and decoding processes and the relevant terms. In section UlI] the 
notation and definitions are introduced. The analytical work is described in section |lVl followed 
by a numerical evaluation of the analytical results in section |Vl Conclusions are drawn in section 
rvTl and proofs of theorems and lemmas are provided in the Appendix. 

IL Background 

In this section an overview of standard LT codes is given, followed by a description of how to 
achieve URT in these codes. Assume we wish to transmit a given amount of data, which is divided 
into k input symbols. An encoded symbol, also called an output symbol, is generated as the bitwise 
XOR of i input symbols, where i is found by sampling the degree distribution, 7r(i). The value i is 
referred to as the degree of the output symbol, and all input symbols contained in an output symbol 
are called neighbors of the output symbol. The degree distribution is a key element in the design 
of good LT codes. The encoding process of an LT code can be broken down into three steps: 

[Encoder] 

1) Randomly choose a degree i by sampling '^{i). 

2) Choose uniformly at random i of the k input symbols. 

3) Perform bitwise XOR of the i chosen input symbols. The resulting symbol is the output 
symbol. 

This process can be iterated as many times as needed, which results in a rateless code. 

A widely used decoder for LT codes is the belief propagation (BP) decoder. The strength of this 
decoder is its very low complexity [2]. It is based on performing the reverse XOR operations from 
the encoding process. Assume a number of symbols have been collected and stored in a buffer, 
which we refer to as the cloud. Then initially, all degree- 1 output symbols are identified, which 
makes it possible to recover their corresponding neighboring input symbols. These are moved to 
a storage referred to as the ripple. Symbols in the ripple are processed one by one, which means 
they are XOR'ed with all output symbols, who have them as neighbors. Once a symbol has been 
processed, it is removed from the ripple and considered decoded. The processing of symbols in the 
ripple will potentially reduce some of the symbols in the cloud to degree one, in which case the 
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neighboring input symbols are recovered and moved to the ripple. This is called a symbol release. 
This makes it possible for the decoder to process symbols continuously in an iterative fashion. 
When a symbol is released, there is a risk that it is already available in the ripple, which means it 
is redundant. This part of the total redundancy is denoted tR. The iterative decoding process can 
be explained in two steps: 
[Decoder] 

1) Identify all degree- 1 symbols and add the neighboring input symbols to the ripple. 

2) Process a symbol from the ripple and remove it afterwards. Go to step 1. 

Decoding is successful when all input symbols have been recovered. If at any point before this, 
the ripple size equals zero, decoding has failed. If this happens, the receiver can either signal a 
decoding failure to the transmitter, or wait for more output symbols. In the latter case, new incoming 
output symbols are initially stripped for already recovered input symbols at the receiver, leaving the 
output symbol with what we refer to as a reduced degree. For symbols having a reduced degree, 
i is referred to as the original degree. If the reduced degree is one, the symbol is added to the 
ripple and the iterative decoding process is restarted. In case the reduced degree is greater than one, 
the symbol is added to the cloud, while a reduced degree equal to zero means that the symbol is 
redundant. This part of the total redundancy, e, is denoted cq, and we have that e = eo + ej?. 

A. LT Codes with URT 

In this work we will use the approach to URT proposed in [fT2]. In this approach the uniform 
distribution used for selection of input symbols is replaced by a distribution which favors more 
important symbols. Hence, in step 2 of the encoder, a non-uniform random selection of symbols is 
performed instead. This solution to URT has no impact on the decoder. We refer to these codes as 
URT-LT codes. 

III. Definitions and Notation 

Vectors will be denoted in bold and indexed with subscripts, e.g. Xj is the i'th element of the 
vector X. The sum of all elements is denoted X and the zero vector is denoted 0. Random variables 
are denoted with upper case letters and any realization in lower case. The probability mass function 
(pmf) of a random variable X is denoted fxix). For ease of notation, we will denote the conditional 
distribution of X given Y as fxix\y) as an equivalent of fx{x\Y = y). 

March 2, 2013 DRAFT 



5 



In URT-LT codes, the k input symbols are divided into subsets Si, S2,..., sn, each with size 
aik, a2k,..., ajyk, where J2f=i^j — 1- refer to these subsets as layers. We define the vector 
a = [«!, 0:2, Oat]. The probability of selecting input symbols from Sj is pj{k)ajk, such that 
'^j'=iPj{k)ajk = 1 and without loss of generality we assume that Pi{k) > Pj{k) if i < j. Note 
that if Pj{k) = J Vj, then all data is treated equally, as in the standard single layer LT code. We 
define a vector, B, where /3, = 

An encoded symbol of a URT-LT code can be seen as having at most dimensions. The A^- 
dimensional original degree is denoted j, where j„ denotes the number of neighbors belonging to 
the n'th layer. Correspondingly, we refer to i' as the reduced degree, where i'„ denotes the reduced 
number of neighbors belonging to the n'th layer. Moreover, we define L, where L„ denotes the 
number of unprocessed input symbols from the n'th layer. Similarly, we define R, where i?„ is 
the number of symbols in the ripple belonging to the n'th layer. The cloud content is denoted C, 
where Ci is the number of symbols in the cloud having original degree i. Note that no differentiation 
between layers is made for the cloud. For the purpose of the analysis, this differentiation is not 
necessary in the definition of the cloud content. Instead, the differentiation will be made in the 
analysis. By Ji, we denote the set of j which satisfy jn > i'n, n = 1,2, N, and j = i. 

Definition 1. (Decoder State) A decoder state, D, is defined by three parameters; the remaining 
unprocessed symbols, L = [Li L2...Ln], the ripple content, R = [Ri R2...Rn], and the cloud 
content, C = [C2 Cs-.-Ck]- Hence, 

D=[LR C]. 

The receiver collects a number of encoded symbols, denoted A, prior to decoding. We define 
the vector Q, = [Qi, . . . , fij, . . . , Vlk], where denotes the number of symbols with original degree 
i among the A collected symbols. After having identified all ^li degree- 1 symbols and created the 
initial ripple, we have what we refer to as an initial state, whose distribution function is defined in 
Definition |2l 

Definition 2. (Initial State) An initial state, = [L^ R^ C^], is defined as the state of the decoder 
after having identified the initial ripple, but before processing the first symbol. Its probability 
distribution function is denoted fjji (</^| A) and is supported by the state space D. By X, we denote 
the set of all for which fj)i (d^ | A) > 0. 
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Example: Consider the case oi k = 10, = 2, ai/E = 6 and 0:2 A; = 4. Decoding is at- 
tempted at A = 10 and the received output symbols have the following degrees respectively: 
2, 3, 2, 4, 7, 1, 2, 1, 4, 1. Hence, Vt = [3, 3, 1, 2, 0, 0, 1, 0, 0, 0]. The three degree-1 symbols constitute 
the initial ripple and two of them belong to layer 1. In this case the initial state will be as follows: 



= [6 4], 
R' = [2 1], 

= [3 1 2 1 0] . (1) 

With an initial state as a starting point, the decoding process can be performed. Whenever a 
symbol from the ripple is processed, L will decrease by one. We refer to this as a decoding step. A 
new decoding step can only be performed if the ripple size is greater than zero. The state distribution 
after k — L decoding steps, given that A output symbols have been collected prior to decoding, is 
denoted /^f, (^d^\A^. It is implicitly understood that the A output symbols, through sampling of 
the degree distribution, give rise to the initial state distribution, fi^i (d^|A), which is the starting 
point of the decoding. A recursive expression of fj-^t (^d^\^^ is presented in equation (|2]), where 
L is the recursion parameter. The joint state distribution is in Q expressed as a function of the 
individual conditional distribution functions. Here denotes the decoder state when L symbols 
remain unprocessed. Through the definition of the decoder state, we similarly have L^, and 
C^. Note that fixing L only fixes the sum of L, thereby leaving as a random variable. 



fj,t(d^\A)=fDi(d^\A), for/GX, (2) 



where /^i (-I-), /^i (-I-) and /^^ (-I-) denote conditional distributions of R , C and L^, respec- 
tively, and are derived in the analysis in section JV] 

In a decoding step, a number of output symbols may release and enable recovery of input 



symbols. We dclinc the matrix M'' = M{\ . . . , M'\ M 



, with column vectors Mf 
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Mfl, . . . , M^j, . . . , Mj^j , where M^^ denotes the number of releases in the {k — L)'th decoding 



step from symbols of original degree i, whose single remaining neighbor belongs to layer n. 
Similarly, we define the row vectors of matrix as = M^^, . . . , M^^, . . . , M^^ . Whenever 
ripple size equals zero, the decoding process stops. In this case we are left with what we refer to 
as a terminal state. Its distribution function is defined in Definition |3l 

Definition 3. (Terminal State Distribution) A terminal state, = [L"^ C^], is defined as a 
state in which R = 0. Hence, 

. ^ X \ fnt (^i^|A) , for = 0, 
I 0, elsewhere. 

If the decoder is in a terminal state and all k input symbols have not yet been decoded, it means 
that more symbols must be collected in order to further progress the decoding. Once a symbol of 
reduced degree 1 is received, the decoding can be restarted. The number of symbols collected while 
being in the terminal state is denoted A^t. 

Table H] shows an example of a decoder state evolution, where each row refers to a decoder state, 
in which k — L symbols have been decoded and processed. The initial state is the example from 
O. The first decoding attempt results in a terminal state after processing three symbols. In this 
terminal state, two new symbols of reduced degrees 2 and 1 respectively are received, which enables 
further progress until a new terminal state is reached. Here a single new symbol of reduced degree 
1 is collected, which enables decoding of the final symbols. The individual decoding attempts are 
indicated with double line separations, and terminal states are highlighted in bold. Note that a 
decoding attempt can be interpreted as a realization of the recursion in (|2]). The evolution of L is 
illustrated in Fig. |2] and the graph representation of this example is illustrated in Fig. [T] 

We have allowed ourselves an abuse of notation, since e.g. the T in f^r (d^|A) does not refer 
to a specific value of L. It is used as an indication of a certain type of state, in this case a terminal 
state. Hence, whenever is used, where X ^ L, this refers to a subset of the state space, for 
which certain criteria are defined. 

IV. Analysis 

When an LT code has been partially decoded, i.e. k' , < k' < k, input symbols have been 
recovered, there is a probability that the reduced degree of a new incoming symbol is zero. If 



March 2, 2013 



DRAFT 



8 

this happens, the symbol is redundant and therefore discarded. Naturally this probability is of great 
importance in the search of well performing LT codes. In this analysis we derive the reduced degree 
distribution for an LT code with URT and use it to show the probability of redundancy in such 
codes. We also show how simple use of ACK can significantly decrease this probability. 



N 



A. URT-LT Codes Without Feedback 

Initially, we express in Theorem [H the reduced degree distribution, ix'^ii' for a certain /3-value 
and as a function of the number of unprocessed symbols from individual layers, t. 

Theorem 1. (N-Layer Reduced Degree Distribution) Given that the encoder applies TT{i) in an 
N-layer URT-LT code with parameters, a and 0, where symbols remain unprocessed from the 
n'th layer, n = 1,2...N, the reduced degree distribution, n'^(i',^), is found as 

\'n'y jn-i'n ) 
( ank\ 

■ ^ \ 1 \ jn J 

jeJi ^ n=l 
for en<i'„<ank, n=l,2,...,N, 

7r^(i',f) = elsewhere. 

where $ is Wallenius' noncentral hypergeometric distribution. □ 

When evaluating Tr'^ii',?.) at t' = we get an interesting quantity. At a given terminal state, 
dF , during transmission, when £^ symbols remain unprocessed from the n'th layer, n = 1,2...N, 
TT^ (0,£^) is the probability that the next received symbol is redundant. This is a key element of 
this analysis, since it enables us to evaluate the expected value of eg. This requires that we know 
how i'^ evolves as A increases. The derivation of this, is the goal of the further analysis. 

In the rest of the analysis, we will treat the case of = 2, where we refer to the layers as 
base layer (n = 1) and refinement layer (n = 2). In Fig. [3l 7r^(0,l) has been plotted as a function 
of Lb and L/j, the number of undecoded symbols from the base layer and the refinement layer, 
respectively. The parameters, as = 0.5, henceforth denoted as a, = I — a, (3 = — = 9 and 

PR 

k = 100 have been chosen. An optimized degree distribution is not provided in [9] and such an 
optimization is out of the scope of this paper, thus we have chosen the Robust Soliton distribution 
(RSD). The RSD is the de facto standard degree distribution for LT codes and was originally 
proposed in [IJ. The plot shows that the probability of redundancy increases faster for decreasing 
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Lb than for decreasing Lr, which is expected since the base layer symbols are more likely to occur 
as neighbors. 

We will now use vr^ (0,^) to derive the expected amount of redundancy, E [keQ^^max)], due to 
symbols having reduced degree zero, when a maximum of A^ax symbols are collected. In order 
to do this, we must find the expected number of symbols received in each possible state. In this 
regard, we note that the A'th symbol is received in state d^, if the decoding of the first A — 1 
symbols resulted in the terminal state dF . Hence, the expected number of symbols, E \/\gr{/\max)], 
received while being in state , equals the expected number of times decoding fails in that state. 
When having E [^gr{/S.rnax)], it is easy to obtain E [A;eo(Amaa)] by multiplying with vr^ (0,1^) and 
summing over all , for which P' ^ 0. Note that tt^ (0,I^) only depends on ^ and not A, hence 
it can be left outside the innermost expectation. The expected amount of redundancy is expressed 
in Theorem |2l 

Theorem 2. {Redundancy in Two-Layer URT-LT Code) In a two-layer URT-LT code using any 
degree distribution, 7r{i), the expected total amount of redundancy, E [keQ{Amax)], due to reduced 
degree zero, is: 

E [keo{Ama.)] = J2 ^ [AaT{Amax)] 4 {0,i^) , 

E[AaT{Ama.)]= J^/dt {d^\A). 

A=l 

□ 

Theorem |2] makes use of the recursive state distribution function in Q. This function depends 
on the initial state distribution, /£>/ (d^|A), which has not yet been derived, as is the case for 
the conditional distributions of the three dimensions of the decoder state, /^^ (-I-), f^i (■!■) and 
fj^i ("I")- To support the derivation of these, we state Lemma [T] which expresses the distribution, 
/a/' (^nl^n) ^n)' of t^c numbcr of symbols added to the n'th dimension of the ripple, 
when the ripple size in this dimension is r„, symbols remain unprocessed in the n'th layer and 
m„ symbols have been released and have a neighbor from the n'th layer. 

Lemma 1. (Ripple Influx) When nin symbols have been released, i.e. have only one neighbor, which 
belongs to the n'th layer, at a point where in symbols remain unprocessed from the n'th layer and 
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the ripple contains r„ symbols in the n'th dimension, the random variable denotes how many 
of those will be added to the ripple. It has the following distribution: 

q=m'„ " 

where T is the hypergeometric distribution. □ 

For the initial state distribution, it is given that = [ak, (1 — a)k], however, and 
are random variables, which depend on the realization, lo, of ft. These realizations follow the 
multinomial distribution. We have that C- = coi for i > 1. Symbols of degree 1 will release 
immediately, hence = cui. Each released symbol will have a base layer symbol as neighbor 
with probability ^g^^ij^fz^, hence the binomial distribution describes the distinction of the 
releases among layers, since = M^^ + M^^. The releases within a layer will potentially result 
in the recovery of a new symbol and thereby contribute to . Lemma [U is thus used to express 
the distribution of given M^^ and M^^. We then get the following initial state distribution: 



X I'm^ {rh\mBi, ak, O) (rjj|m^ - m%^, (1 - a)k, O) , (4) 

where 9 is the binomial distribution, /i is the multinomial distribution and its realizations are 
constrained by A = rti^ + . 

Lemma [2] introduces /^i, which expresses the probability that the next processed 

symbol is a base layer symbol and the probability that it is a refinement layer symbol. 

Lemma 2. (Next Processed Symbol) Given a decoder state d^^^, the probability that the next 
processed symbol is a base layer symbol = — 1, = ^^^^ j and the probability that it 
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is a refinement layer symbol (^i^ = i^'^^, = i^'^ — ij, is: 



' B 



for = and £ii = - 1. 



L+i , L+i' - "-B ^^^^ ^i? - "-R 



□ 



Next step is to find tlie distribution of the releases during decoding, since they determine the 
dynamics of the buffer content and are the basis of the ripple influx. Lemma [3] expresses the 
probability that a symbol of degree i is released in a certain decoding step. 

Lemma 3. (Release Probability in Two-Layer URT-LT Code) In a two-layer URT-LT code with 
parameters a, /3 and k, the prior probability, qxY, that a symbol of degree i is released as a 
symbol from layer Y, after processing a symbol from layer X, when is and Ir symbols remain 
unprocessed from base layer and refinement layer respectively, is: 



j=0 



j=0 \ \ j ) \ i-j ) 

I / £ ^ak-eB\ Kl~a)k~-lR~l\ 



j=0 



qRRM = J2 "^O'^'"^'/^) Lkyii-1)L ' 

where $ is Wallenius' noncentral hypergeometric distribution. □ 

When symbols are released in the {k — L)'th decoding step, we have that = C/^^^ — M^, 
which allows us to express the cloud development recursively, if we derive the distribution of 
M/". This can be done using Lemma [3] and the binomial distribution, which gives us the cloud 
development expressed in Lemma |4] 

Lemma 4. (Cloud Development in a Decoding Step) Given a decoder state d^~^^, with a cloud 
content c^+\ the probability of having a cloud content ofc^, after processing a symbol from either 
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the base layer (^i^ = £^+^ - 1, 4 = 4^^) or the refinement layer (^4 = = 4^^ - l)' 



1 and 4 = 4+\ 



' (^,^^) + (^,1^) for 4 = 4^' - 

^ Qrb + qRR for 4 = 4^' and 4 = 4+' - 1, 



where 9 is the binomial distribution, q is the probability that an output symbol of degree 

is released when L = £^ and qxy ^fe given in Lemma \3\ 



□ 



Lemma |4] expresses the probability distribution of the cloud and thereby the probability distribu- 
tion of the symbol releases in a decoding step. However, a released symbol is not guaranteed to be 
added to the ripple. Some released symbols might be identical and some might already be in the 
ripple. Lemma [H can be applied to express the distribution of the number of symbols added to the 
two dimensions of the ripple. For this purpose, we introduce qxb which is the probability 

that a symbol releases as a base layer symbol, given that it has been released. It is given by: 



qxB u 



^. for 4 = 4""' - 1 and 4 = 4^S 



(5) 



IRE 



. .f^ , .r< for 4 = 4"^^ and 4 = 4+^ - 1. 

L qRB{i,e^)+qRR{i,e^) B B R R 



We now note that Mj^- follows the binomial distribution with parameters mf and qxB yh^^j, 
i.e. 9 ^m|j,mf,gxB (^h^^^^- Moreover, we have that the distribution of is a convolution of 
these binomial distributions over all i. Hence, 



/ 



B\ 



i=l 



^Bi^^i ,QXB 



(6) 
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k 

where 9 is the binomial distribution and denotes a A;-way convolution of the binomial distribu- 

i=l 

tions with i = 1, 2, k. 

We denote the number of base (refinement) layer symbols, which are added to the ripple, 
^M^'j, and can then express the probability distribution of the ripple size after the processing of 
a new symbol in Lemma [51 

Lemma 5. (Ripple Development in a Decoding Step) Given a decoder state d^'^^, containing a 
ripple size of r^+\ the probability of having a ripple size of r^, after the next decoding step 
resulting in the cloud c^, is: 

X fM^ {^r\^' - ^i4,4^^ - (4^' - 4)) 

4 = ml' +4+^ -(4-^^ -4), 

4 = -fe'+4^^-(4^^-4)- 

□ 

Lemmas \T\ through |5] provide the necessary support for Theorem [2l The case where feedback is 
applied is analyzed in the following subsection. 

B. URT-LT Codes With Feedback 

In this subsection we treat the case where an intermediate feedback message is applied during 
the transmission. This message tells the transmitter that the base layer has been decoded, i.e. it 
works as an acknowledgment of the base layer. The transmitter adapts by excluding the base layer 
symbols from the random selection in step 2 of the encoder. This means that only refinement layer 
symbols are included in future encoding. The feedback message is assumed to be perfect, i.e. zero 
error probability and delay. In the event where the refinement layer is decoded before the base 
layer, no intermediate feedback is transmitted. 

In the case of feedback, we can divide the transmission into two phases; one before feedback 
(phase 1) and one after (phase 2). The number of symbols collected in phase 1 is denoted and 



March 2, 2013 



DRAFT 



14 



the total number of symbols collected in both phases is denoted A^. Hence, the number of symbols 
collected in phase 2 is A^ — A^ One of the main differences between the two phases is the initial 
state distribution. In phase 1 it is equivalent to the case without feedback, whereas in phase 2 the 
initial state will be the result of adding A^ — A^ symbols to the outcome of phase 1 and identifying 
the new ripple. Moreover, in phase 2 the encoder only considers refinement layer symbols, which is 
the equivalent of /3 = 0, thus entailing the reduced degree distribution vtq (0,f^). Phase 1 continues 
as long as 7^ and phase 2 continues as long as £^ 7^ 0. 

The expected redundancy, caused by symbols of reduced degree zero, is denoted E [ke^ (Amax)] 
and is found using the same approach as in Theorem |2l First we find the expected number of 
times the decoder fails in any state in both phase 1 and phase 2. This will provide the expected 
numbers, E [A^T(Amax)] and E [A^T(Am„x)] , of symbols received in any such state in the two 
phases. We then multiply with the probability that the next symbol is redundant, vr^j (0,1^), and 
sum over all terminal states for which 7^ in phase 1, since this is required for phase 1 to 
continue. Similarly, we multiply with ttq (0,1'^) and sum over all d^ for which £]J 7^ in phase 2. 
The expected redundancy is formally expressed in Theorem |3] 

Theorem 3. (Redundancy in Two-Layer URT-LT Code with Feedback) In a two-layer URT-LT code 
using any degree distribution, n^i), and a single feedback message when the base layer has been 
decoded, the expected total amount of redundancy, E \ke^ {/S.max)\, due to reduced degree zero, is: 

E [ke^ (A^,,)] = ^ [A^.(A^,,)] 7c'^ (0,£^) + E [Alr{A^ax)] 7r(, (0,£^) , 

^raax 

E[Ai.(A„,.)] = fnT{d^\A'), 

Ai=l 

E [Alr{A^,x)] = Y E {d^'l^'^^') > 

Ai = l A2=Ai 

where f^r (<i^|A^) is the terminal state distribution in phase 1, which has initial state distribution 
fjjii (d^^|A^), and f^r (d'^IA^, A^) is the terminal state distribution in phase 2, which has initial 
state distribution fj)i2 (d^^|A^, A^). □ 

The initial state distribution for the first phase, fjji^ (d^^|A^), is found using (HJ. For the second 
phase, we note that the initial state distribution, fjji^ (<i^^|A^, A^), will be the result of receiving 
an additional A^ — A^ symbols and identifying the initial ripple, while being in a state where the 
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feedback was transmitted, which is referred to as a feedback state and denoted — [L^ . 
The outcome leading to an initial state for phase 2 can be viewed as the combination of four events. 

El, E2, E3 and £^4, which are defined below: 

El : The terminal state of the first phase is a feedback state, D^. 

E2 : The new — symbols have original degrees as expressed by u^, where ui^ is the 

number of new symbols of original degree i. 
E^ : For each i, out of the u!"^ new symbols of original degree i, m^^- have reduced degree 1, 

and cf + ojf — m% — have reduced degree 0. 
£■4 : Out of the released symbols, rj| are added to the ripple. 

These events are not independent, since £'3 depends on Ei and E2, and £4 depends on Ei, E2 
and £3. We can express the initial state distribution for phase 2 as follows: 

Z^., (rf'^|A\A') = ^ J2 5] P''(^i)P''(^2)Pr(£3|£^i,^2)Pr(£4|£i,£2,£3) 
Si^{d^:cf<ct V^}, 

^2 ^ {u^ ■.00^ = A' -A' and cf + cu^ > cf , Vz}, 

Ss ^ : m'i > r'^ and m% < Vi}, (7) 

where the sets Si, S2 and S3 refers to all possible events Ei, E2 and E3, which enable an event 
£4 that provides d^^. 

The probability of Ei is found by evaluating the feedback state distribution, /df (rf^|A^). A 
feedback state is defined as a terminal state in which Lb — 0, provided that the decoding attempt 
of A^ — 1 symbols resulted in a terminal state in which Lb > 0. Hence, the A'th received symbol 
must have reduced degree 1 and enable the decoder to recover the remaining base layer symbols. By 
/df- (d^^l A^), we denote the state distribution after the decoding of A^ — 1 symbols and receiving 
the A'th symbol, which is potentially added to the ripple. We can then express the feedback state 
distribution, and thereby the probability of Ei, as follows: 
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fDT{d^\A^), for£g = 0, 
0, elsewhere, 



(8) 



where the initial state distribution of the state recursion used to express f^T ((i^|A^) is given by 
foF- (c^^ |A^), which is found as follows: 



0, elsewhere. 
The probability of E2 is found using the multinomial distribution, since the original degrees are 
sampled from 7r(i). Hence, 



Pr(E2)=/i(a;^A2-A^7^(^)). (10) 

£"3 concems the reduced degree, i'j^, which follows the hypergeometric distribution, T(i'^, (1 — 
a)k,i^,i), since new output symbols are encoded using only refinement layer symbols. The 
probability that a symbol with original degree i has reduced degree (1) is denoted po{i) {pi{i)). 
Then for any i, mj|- follows the binomial distribution, 6 (^m^j^^,ujf,pi{i)). The remaining oof — 
m^^- symbols have reduced degree with conditional probability For a given and 

m^^, we have that the number of symbols of reduced degree must be cf + ujf — m^^ — c[^, 
since the total amount of symbols with original degree i is cf + ujf. The probability of this is 
9 {cf + ujf — mj|- — cl'^,ujf — m^j^^, jz^^ j . The probability of E3 is then found as a multiplication 
of the two binomials for all i. Hence, 

Fr{E,\E,, E,) = \{[e {m%, ul 6 (cf + - m% - c^ - J^^) ) • d D 

For a given i?3, the vector mj| is given and thereby m^, the total amount of released symbols. 
The probability that r^^ of these are added to the ripple is found using Lemma [T] Hence, 



V,{E^\EuE2,E^) = Ui, {r'j!\rn'i,4,0) . (12) 
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This concludes the analysis and we are now able to evaluate the presented theorems. This is 
described in the following section. 

V. Numerical Results 

In this section we present evaluations of the expressions derived in section |IV] as well as 
simulations of a URT-LT code. All evaluations are performed with k = 100, a = 0.5 and the 
RSD as the degree distribution, with parameters c = 0.1 and 5 = 1. To evaluate the expressions, 
we use Monte Carlo simulations with 1000 iterations. 

Initially, we evaluate E [A^r^Amax)], i-e- the expected amount of symbols received in a given 
terminal state, d^, at different values of /3 and in the case of no feedback. We evaluate it at all 
possible and normalize with the maximum number of received symbols, Amax, hence providing 
the expected fraction of symbols received in that given state. Moreover we marginalize out and 
C^, which means we get the expected fraction as a function of L^. The results are illustrated in 
Fig. |4] for (3 = {1, 4, 16, 32}. Note that the color code is using a logarithmic scale, in order to better 
visualize the results. From the figure it is seen that at /3 = 1, symbol receptions are distributed 
symmetrically around the line = L^, which was expected, since at /3 = 1 we have a standard LT 
code with no bias towards the base layer. In this case, we also see that most symbols are received 
in states where none or very few input symbols have been recovered. This confirms the well-known 
avalanche effect in LT decoding llT3l . which refers to the fact that the first many received symbols 
only enable the recovery of very few input symbols. Then suddenly, a single new symbol enables 
the recovery of all the remaining input symbols. A brief look at Fig. [3] reveals that this effect is 
essential to the performance of standard LT codes. Moving on to higher (3 values, we see the bias 
towards the base layer come into effect. Clearly, symbols are more likely to be received in states 
where < L^, which is an indication of the URT property. However, it is also clear that this bias 
results in more symbols being received in states where = and > 0. In other words, the 
avalanche fades out prematurely and new symbols must be received in a state where few symbols 
are unrecovered, leading to high probability of redundancy, cf. Fig. [3l The case of /3 = 4 with 
three simulations, not analytical results, of the corresponding URT-LT code added as an extra layer 
on top is shown in Fig. [51 In the simulations Lb and Lr have been logged during a successful 
decoding. This figure illustrates that the behavior seen in Fig. |4] corresponds well with practice. 
The fact that the URT property is indeed achieved with these codes is verified by Fig. |6l where 
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the probability of having successfully decoded the base layer is plotted as a function of A. Note 
that this probability is found by evaluating /£)t (d^, A) at = and marginalizing out all other 
dimensions of the state space. 

Next, we evaluate E [keo{Amax)] and E [keQ (Amax)] ■> again for increasing values of (3. The results 
are shown in Fig. |7](no feedback) and Fig. [8] (with feedback) as a function of A^ax- In general in 
both figures, we see that the amount of redundancy remains close to zero until roughly 50 symbols 
have been received, after which redundancy starts to occur. The amount of redundancy increases 
until all input symbols have been recovered (L^ = 0) with high probability and converges to a 
level, that depends upon (3. In the case of no feedback, it is seen that the redundancy increases 
linearly with (3. Hence, the URT property comes at a significant price in the form of additional 
overhead and this price increases with the bias towards the base layer. In the case where feedback 
is used to acknowledge the base layer, we also see an increase in redundancy for increasing (3. 
However, the redundancy reaches a maximum at /3 = 8 and then starts to decrease for /3 > 8. This 
clearly demonstrates the great potential a single intermediate feedback has in URT-LT codes. Fig. [9] 
shows the converged redundancy values, i.e. E [A;eo(c>o)] and E [/ce^(oo)], as a function of (3. This 
makes it easy to compare the two schemes. Note that the redundancy converges for increasing 13 
in the scheme applying intermediate feedback. This is due to the fact that this scheme, in the limit 
/3 = oo, is the equivalent of two standard LT coded transmissions, with ki = ak and k2 = (1 — a)k. 

Finally, we present an evaluation where we map terminal states to distortion measures. We assume 
that the decoded data describes a successive refinable [|T4l Gaussian source with unit variance, which 
has been partitioned into two layers; base layer and refinement layer. It is assumed that if both 
layers are decoded {L^ = L'^ = 0), 1 bit/sample is available to describe the source. If only the base 
layer is decoded {L^ = 0, > 0), a bit/sample is available. If either no layers (L^ = > 0) 
or the refinement layer only (L^ > 0, = 0) is decoded, we have bit/sample to describe the 
source. This layered structure, which is common in e.g. video streaming, motivates the use of URT- 
LT codes, since the base layer has more importance with respect to the distortion measure. For the 
unit variance Gaussian source, we have that D > 2^^'^^\ where D is the distortion measured as 
the mean squared error and R is the rate measured in bits per sample. We assume that the bound 
is achievable and use this relationship to calculate the expected distortion of the URT-LT codes 
for both the case with acknowledgment of the base layer and the case without. The results are 
shown in Fig. [10] for the same code parameters as in previous evaluations. The figure shows that 
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the optimal value of /3 depends on the overhead of the URT-LT code. Moreover, it is evident that 
in the case of no acknowledgment of the base layer, if a low distortion is required at low overhead, 
the price to pay is a very significant increase of the distortion at higher overhead. If the base layer 
is acknowledged, the distortion quickly decreases to the minimum level, regardless of the choice 
of /3. 

VI. Conclusions 

We have analyzed finite-length LT codes with unequal recovery time, termed URT-LT codes 
in this paper. The analysis is based on a state recursion function, which allows us to evaluate 
the distributions of the ripple, cloud and decoding progress in individual segments, as the overall 
decoding progresses. This gives novel insight into the probabilistic mechanisms in URT-LT codes, 
which is a major contribution of this paper. The analysis enables us to evaluate the expected amount 
of symbols with reduced degree 0, i.e. redundant symbols, during a transmission. Evaluations in 
the case of two data segments show that this amount increases roughly linearly with the level of 
priority given to the first data segment. As a result, successful decoding of the lower prioritized 
data is delayed substantially. Thus, we can conclude that the unequal recovery time comes at a 
significant price in terms of redundancy in lower prioritized data. 

A slight modification of the URT-LT codes has been proposed, where an intermediate feedback 
message informs the encoder that the higher prioritized data has been decoded. The encoder adapts 
by excluding the decoded data from future encoding. Analysis of this code reveals that such a 
modification is able to dramatically decrease the redundancy in lower prioritized data. The impact 
of this improvement is further illustrated with an evaluation, where the decoding probabilities are 
mapped to an expected distortion measure of a layered unit variance Gaussian source. Improvements 
of roughly 0.25 in mean squared error is observed compared to the original URT-LT code. 

Appendix 

Proof of Theorem [71- Initially it is noted that j follows Wallenius' multivariate noncentral 
hypergeometric distribution with parameters i, ak and /3. This distribution generalizes the hyper- 
geometric distribution to take nonuniform sampling into account. For a certain j, the probability 
of receiving a symbol with reduced degree i' is found as a product of N regular hypergeometric 
distributions. This follows from the fact that a degree reduction from j„ to i'^, n = 1,2, ...,N, 
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occurs when i'^ of the neighbors are among the undecoded symbols and the remaining jVi — C 
neighbors are among the q;„A; — already decoded symbols. Sampling of neighbors within a single 
layer is done uniformly, hence the hypergeometric distribution applies. Finally it is noted that any 
symbol with j ^ Ji can potentially reduce to i' since j„ > n = 1,2, A^, and that Ji = % 
unless i' < i < i' + k — L. ■ 
Proof of Theorem Wi The probability of ending in the terminal state , thus receiving the 
next symbol in this state, when trying to decode the first A symbols, is given by f^r (dF , A). The 
expected number of symbols, E [ArfT(Ama^)], received in this state during an entire transmission 
is found by summing /£)t (d^. A) for all possible A, i.e. A = 1, A^a^. Of these, an expected 
E [AdT(Amaa;)] TT^ (0,l^) havc rcduccd degree 0. Finally, the total amount of symbols, E [eoA;], with 
reduced degree is found by summing over all terminal states in which the transmission continues, 
i.e. any state in which 7^ 0. ■ 
Proof of Theorem^ The contribution from phase 1, X^d^^i^^o^ (O;^^)' follows 

the same structure as Theorem |2l although with a different condition for receiving more symbols, 
since phase 1 continues as long as £^ 7^ 0. The same holds for the contribution from phase 2, 
^^T.£T ^qE rA^r(oo)l ttq (0,1"^), although the amount of symbols received in phase 2 is A2 — Ai 
and the condition for receiving more symbols is 7^ 0. See proof of Theorem |2]for details. ■ 
Proof of Lemma [7]- If m„ samples are drawn uniformly at random from a set of size £„, 
then the number of unique samples, g, will follow a distribution expressed by (f^frrr"' where 
Zq{rhn) = Yll=o — 1)^. This is an inverse variant of the coupon collector's problem, 

whose details can be found in []. Having q unique released symbols, the number of those, m^, who 
are among the — r„ symbols not already in the ripple follows the hypergeometric distribution. 
Having ripple additions can be the result of any amount of unique releases higher than m^. ■ 
Proof of Lemma |2]- Since the next symbol to be processed is chosen uniformly at random 
among the symbols in the ripple, the probability that it is a base layer symbol is the fraction of base 
layer symbols currently in the ripple, . Similarly, the probability that it is a refinement 



layer symbol is -j—f- 



L+l.L+l ■ 
B R 



Proof of Lemma\3\ Given a degree i, afc, (3) expresses the probability of having j base 

layer symbols among the i neighbors. See proof of Theorem [Tlfor details. Assuming a base layer 
symbol has just been processed, a symbol is released as a new base layer symbol when £5 and £r 
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symbols remain unprocessed from base layer and refinement layer respectively, if j — 2 base layer 
neighbors are among the ak — is — I first processed base layer symbols, one is the {ak — isYth 
processed base layer symbol, all i — j refinement layer neighbors are among the (1 — a)k — iji 
processed refinement layer symbols and the last base layer neighbor is among the Eb undecoded 
base layer symbols. This proves the expression for qBsih^'^)- Similar proofs can be made for 
QBRiiJ^), qRBiiJ^) and qRniiJ^). ■ 
Proof of Lemma Any symbol with original degree i is released in the (k — L)'th decoding 
step with prior probability q according to Lemma |3] All symbols with original degree i still 

left in the cloud after k — {L+l) decoding steps will release with conditional probability q^ = 

— 1 '^^^'^ \ , where the denominator expresses the remaining probability mass 

of the prior probability distribution from Lemma |3] Hence, the amount of degree i symbols, M/", 
released in the next decoding step, follows the binomial distribution, 9 ^mf , cf^\ qc j . The 

probability of having a cloud of after that decoding step is thus found as a product of binomial 
distributions, evaluated at mf, V i. ■ 
Proof of Lemma \5\- For a certain cloud development in the {k — L)'th decoding step, the total 
amount of releases, rh^, is found as = Yl'i=2 ^f^^ These releases are differentiated among 
layers using equation thereby achieving and = rh^ — m^. Lemma [U is then used to 
express the distributions of the amounts, M^' and M^', which are added to the ripple. 
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Fig. 1. Graph representation of the example code. 
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Fig. 2. An example of a decoder state evolution with two terminal states before successful decoding. 
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Fig. 3. The probability of receiving a symbol with reduced degree zero, vr^ (0,^), for a two-layer LT code with parameters a — 0.5, 
13 = 9 and k = 100. 
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Fig. 6. Probability of successfully decoding the base layer as a function of the maximum amount of collected symbols. 
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Fig. 7. Expected amount of redundancy due to reduced degree zero as a function of the maximum amount of collected symbols 
for different values of P in the case of no feedback. 
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Fig. 8. Expected amount of redundancy due to reduced degree zero as a function of the maximum amount of collected symbols 
for different values of /3 in the case of feedback. 
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Without FB 
^ — With FB 




Fig. 9. Asymptotic values of E [keo{Amax)] and E [ke^ (Amax)] as a function of (3. 
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Fig. 10. Expected distortion of a layered unit-variance Gaussian source as a function of the maximum amount of collected sjmbols. 
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TABLE I 

Example of a decoder state evolution. 
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